1. Field of the Invention
The present invention generally relates to data processing systems and communications systems. More particularly, the invention relates to messaging.
2. Description of the Related Art
Messaging and queuing is a method of inter-program communication which allows programs to send and receive application-specific data without having a direct connection established between them. A message consists of two parts—application data and a message descriptor containing control information. The application data in a message is defined and supplied by the application program which sends the message. There are no constraints on the nature of the data in a message (for example, it could consist of one or more bit strings, character strings, binary integers, etc). Applications view the string of bits and bytes that make up a message as consisting of a sequence of items which each have a particular meaning and data type. In addition to the application data, a message has associated with it some ancillary data. This is information that specifies the properties of the message, and is used by the message queuing service to decide how the message should be processed. Some of this information must be specified by the sending application.
A message queue is a named object in which messages accumulate and from which they are later removed. Each queue belongs to one particular queue manager, which is the system service that provides the message-queuing facilities used by applications. The queue manager is responsible for the maintenance of that queue. When messages are added to a queue, they are added at the end, and when messages are taken from a queue they are normally removed from the front (although facilities do exist for reading messages in other than FIFO order). The physical representation of a message queue depends on the environment, but can be a buffer or buffers in main storage, a file or files on disk or other permanent storage device, or both of these. The physical management of message queues is entirely the responsibility of a queue manager, and such details are not made apparent to application programs. Applications can view a message queue simply as a “black box” in which messages accumulate. Applications have access to message queues by using message queuing API calls—obtaining message queueing services by using the message queueing calls to communicate with the queue manager that is installed on the same system as the application (i.e. the local queue manager).
Applications communicate by agreeing to use particular named message queues, sending messages to the specific target queues that the application programs have agreed to read from. The locations of these queues need not be apparent to the applications which send the messages; each application interacts only with its local queue manager, and it is the network of interconnected queue managers that is responsible for moving the messages to the intended queues. In this way, the message queuing software greatly simplifies the level of complexity that is required of the application programs, removing the need for them to implement their own complex communications controls.
One environment which uses messaging to advantage is a database environment. Databases are computerized information storage and retrieval systems. One common database is known as a relational database management system (RDBMS), which is a computer database management system that uses relational techniques for storing and retrieving data. Databases are computerized information storage and retrieval systems in which data in the form of tables (formally denominated “relations”) are typically stored for use on disk drives or similar mass data stores. A “table” includes a set of rows (formally denominated “tuples” or “records”) spanning several columns. Data in the tables is stored, retrieved and deleted using high-level query languages, such as the Structured Query Language (SQL). Reference is made to C. J. Date, An Introduction to Database Systems, 6th edition, Addison-Wesley Publishing Co. Reading, Mass. (1994) for a comprehensive general treatment of the database art.
Commonly, database management systems (DBMS) are part of a distributed system comprising a plurality of servers and clients, as in the case of the client-server model. Typically, the clients and servers communicate via a local area network (LAN) or wide area network (WAN) using well-known protocols such as Transmission Control Protocol and Internet Protocol (TCP/IP). The clients typically make requests to the servers, which manage one or more databases. Once a response for the client's request is formulated, the response is provided to the client.
One illustrative networked database environment 100 using messaging to advantage is shown in FIG. 1. In general, the networked database environment 100 comprises a sending server 102 in communication with a plurality of receiving servers 1201, 1202, . . . 120N(collectively referred to as the receiving servers 120). The sending server 120 includes a database 104 containing a plurality of tables (not shown). The information contained in the database 104 is accessed by a sending application 106. In general, the sending application 106 accesses the database 104 in the event of a modification of the data contained in the database 104. The sending application 106 then places the information (in the form of a message) on a sending queue 110 of a messaging facility 108. The message is then sent to each of the receiving servers 120 via a dedicated Channel 1121, 1122, . . . 112N established over a network connection 114 (e.g., a TCP/IP connection).
Each of the receiving servers 120 includes a messaging facility 1221, 1222, . . . 122N(collectively referred to as the messaging facilities 122), a custom receiving application 1261, 1262, . . . 126N(collectively referred to as the custom receiving applications 126) and a custom database 1281, 1282, . . . 128N(collectively referred to as the custom databases 128). Incoming messages from the sending server 102 are received on a receiving queue 124 of the messaging facility 122. The messages may then be retrieved from the receiving queue 124 by the custom receiving application 126. Ultimately, the messages are written to the custom database 128 to update the information contained therein.
In a typical implementation, the sending server 102 is a data warehouse configured to send data changes to the receiving servers 120, which are datamarts. For example, the data warehouse may be a master repository of information for a plurality of products, while each datamart maintains information for only selected products. As a result, each receiving server 120 (i.e., each datamart) may only be interested in a portion of a data change. Accordingly, each custom receiving application 126 is configured to extract and store only selected information (i.e., information specific to a product(s) of interest to the datamart).
Occasionally, the message format of the message containing the data changes is itself changed. Accordingly, a developer/administrator (a human being) of the sending server 102 is responsible for defining a new common message format and then providing the new common message format to the receiving servers 120. This management role of the sending server developer/administrator is complicated by the fact that each of the receiving servers 120 has its own custom receiving application 126. In the most common implementation, the developer/administrator of the sending server 102 engages each developer (a human being) of the receiving servers 120 to review the message format. The message format is provided to the developer of each receiving server 120 via electronic mail, conventional mail or some other similar service. Each receiving server developer then develops and tests a custom receiving client that complies with the message format. Once the clients have been successfully configured, the receiving servers 120 may begin receiving data updates from the sending server 102. Each update to the data warehouse generates a plurality of messages, a single message for each datamart receiving client. Upon receiving the message, each receiving client extracts data specific to their datamart application.
Accordingly, the message format distribution, the custom development, and the testing each involve manual operations that contribute to inefficient operation. As a result, it would be preferable to eliminate or reduce the overhead involved in distribution, development, and testing.
Therefore, there is a need for an improved method and system for messaging management.